Image processing device, image processing system, image display method, and image processing program

ABSTRACT

An image processing device for generating and updating three-dimensional data representing a biological tissue based on tomographic data of the biological tissue acquired by a sensor that acquires the tomographic data while moving through a lumen of the biological tissue, and causing a display to display the three-dimensional data as a three-dimensional image includes: a control unit configured to color, in the three-dimensional image, at least a voxel representing an inner surface of the biological tissue or a voxel that is adjacent to the voxel representing the inner surface and that represents the lumen among a first voxel group corresponding to a cross section indicated by tomographic data newly acquired by the sensor, in a manner of being distinguished from a second voxel group corresponding to another cross section of the biological tissue.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/JP2021/035460 filed on Sep. 27, 2021, which claims priority toJapanese Patent Application No. 2020-164185 filed on Sep. 29, 2020, theentire content of which is incorporated herein by reference.

TECHNOLOGICAL FIELD

The present disclosure generally relates to an image processing device,an image processing system, an image display method, and an imageprocessing program.

BACKGROUND DISCUSSION

U.S. Pat. Application Publication No. 2010/0215238 A1, U.S. Pat. No.6,385,332 B, and U.S. Pat. No. 6,251,072 B disclose a technique ofgenerating a three-dimensional image of a cardiac cavity or a bloodvessel by using an ultrasound (US) image system.

A treatment using intravascular ultrasound (IVUS) is widely executed onregions such as cardiac cavity, cardiac blood vessel, and lower limbartery. The IVUS is a device or method for providing a two-dimensionalimage of a plane perpendicular to a long axis of a catheter.

At present, an operator needs to execute a treatment whilereconstructing a three-dimensional structure by stacking two-dimensionalimages of IVUS in one’s head, which causes a barrier particularly toyoung doctors or inexperienced doctors. In order to eliminate such abarrier, it is conceivable to automatically generate a three-dimensionalimage expressing a structure of a biological tissue such as a cardiaccavity or a blood vessel from the two-dimensional images of IVUS and todisplay the generated three-dimensional image toward the operator.

However, IVUS has a restriction that only information on one crosssection can be obtained. Even if the structure of the biological tissueis expressed in a three-dimensional image, true real time information isonly information on one cross section. Therefore, it is required thatthe operator understand which part in the three-dimensional imageinformation currently obtained by IVUS, that is, the latest informationcorresponds to.

SUMMARY

The present disclosure is to indicate which part in a three-dimensionalimage a cross section of a biological tissue indicated by tomographicdata newly acquired by a sensor corresponds to.

An image processing device according to one aspect of the presentdisclosure is an image processing device for generating and updatingthree-dimensional data representing a biological tissue based ontomographic data of the biological tissue acquired by a sensor thatacquires the tomographic data while moving through a lumen of thebiological tissue, and causing a display to display thethree-dimensional data as a three-dimensional image. The imageprocessing device includes: a control unit configured to color, in thethree-dimensional image, at least a voxel representing an inner surfaceof the biological tissue or a voxel that is adjacent to the voxelrepresenting the inner surface and that represents the lumen among afirst voxel group corresponding to a cross section indicated bytomographic data newly acquired by the sensor, in a manner of beingdistinguished from a second voxel group corresponding to another crosssection of the biological tissue.

In one embodiment, the control unit is configured to color all voxelsrepresenting the biological tissue among the first voxel group in amanner of being distinguished from the second voxel group.

In one embodiment, the control unit is configured to color not only thefirst voxel group but also at least a voxel representing the innersurface or a voxel that is adjacent to the voxel representing the innersurface and that represents the lumen among a voxel group correspondingto a cross section adjacent to the cross section corresponding to thefirst voxel group, in a manner of being distinguished from a voxel groupcorresponding to another cross section of the biological tissue.

In one embodiment, the control unit is configured to set a color of atleast the voxel representing the inner surface or a color of the voxelthat is adjacent to the voxel representing the inner surface and thatrepresents the lumen among the first voxel group to a color differentfrom any color of the second voxel group, thereby coloring at least thevoxel representing the inner surface or the voxel that is adjacent tothe voxel representing the inner surface and that represents the lumenamong the first voxel group in a manner of being distinguished from thesecond voxel group.

In one embodiment, the control unit is configured to cause the displayto display, together with the three-dimensional image, a two-dimensionalimage representing the cross section indicated by the tomographic datanewly acquired by the sensor.

In one embodiment, the control unit is configured to combine a firstgraphic element representing a moving range of the sensor with a secondgraphic element representing a position of the sensor and cause thedisplay to display the combination together with the three-dimensionalimage.

In one embodiment, the control unit is configured to cause the displayto display the first graphic element in a direction in which alongitudinal direction of the lumen in the three-dimensional image and along axis direction of the first graphic element are parallel to eachother.

An image processing system according to one aspect of the presentdisclosure includes: the image processing device; and a probe includingthe sensor.

In one embodiment, the image processing system further includes: thedisplay.

An image display method according to one aspect of the presentdisclosure is an image display method for generating and updatingthree-dimensional data representing a biological tissue based ontomographic data of the biological tissue acquired by a sensor thatacquires the tomographic data while moving through a lumen of thebiological tissue, and causing a display to display thethree-dimensional data as a three-dimensional image. The imageprocessing method includes: coloring (a computer coloring), in thethree-dimensional image, at least a voxel representing an inner surfaceof the biological tissue or a voxel that is adjacent to the voxelrepresenting the inner surface and that represents the lumen among afirst voxel group corresponding to a cross section indicated bytomographic data newly acquired by the sensor, in a manner of beingdistinguished from a second voxel group corresponding to another crosssection of the biological tissue.

A non-transitory computer readable medium storing an image processingprogram according to one aspect of the present disclosure when executedby a computer, generates and updates three-dimensional data representinga biological tissue based on tomographic data of the biological tissueacquired by a sensor that acquires the tomographic data while movingthrough a lumen of the biological tissue, and causes a display todisplay the three-dimensional data as a three-dimensional image, andperforms processing comprising: coloring, in the three-dimensionalimage, at least a voxel representing an inner surface of the biologicaltissue or a voxel that is adjacent to the voxel representing the innersurface and that represents the lumen among a first voxel groupcorresponding to a cross section indicated by tomographic data newlyacquired by the sensor, in a manner of being distinguished from a secondvoxel group corresponding to another cross section of the biologicaltissue.

According to the present disclosure, it is possible to indicate whichpart in the three-dimensional image the cross section of the biologicaltissue indicated by the tomographic data newly acquired by the sensorcorresponds to.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of an image processing system according toan embodiment of the present disclosure.

FIG. 2 is a diagram showing an example of a screen displayed on adisplay by an image processing system according to the embodiment of thepresent disclosure.

FIG. 3 is a diagram showing an example of a two-dimensional imagedisplayed on the display by the image processing system according to theembodiment of the present disclosure.

FIG. 4 is a diagram showing an example of a cutting region formed by theimage processing system according to the embodiment of the presentdisclosure.

FIG. 5 is a perspective view of a probe and a drive unit according tothe embodiment of the present disclosure.

FIG. 6 is a block diagram showing a configuration of an image processingdevice according to the embodiment of the present disclosure.

FIG. 7 is a flowchart illustrating an operation of the image processingsystem according to the embodiment of the present disclosure.

FIG. 8 is a flowchart illustrating an operation of the image processingsystem according to the embodiment of the present disclosure.

FIG. 9 is a diagram showing a result of binarizing a cross-sectionalimage of a biological tissue in the embodiment of the presentdisclosure.

FIG. 10 is a diagram showing a result of extracting a point group on aninner surface of the biological tissue in the embodiment of the presentdisclosure.

FIG. 11 is a diagram showing a result of calculating a centroid positionof a cross section of the biological tissue in the embodiment of thepresent disclosure.

FIG. 12 is a diagram showing results of calculating centroid positionsof a plurality of cross sections of the biological tissue in theembodiment of the present disclosure.

FIG. 13 is a diagram showing a result of smoothing the results in FIG.12 .

FIG. 14 is a diagram showing an example of a screen displayed on adisplay by an image processing system according to one modification ofthe embodiment of the present disclosure.

DETAILED DESCRIPTION

Set forth below with reference to the accompanying drawings is adetailed description of embodiments of an image processing device, animage processing system, an image display method, and an imageprocessing program.

In the drawings, the same or corresponding parts are denoted by the samereference numerals. In the description of the present embodiment, thedescription of the same or corresponding parts will be omitted orsimplified as appropriate.

An outline of the present embodiment will be described with reference toFIGS. 1 to 4 and FIG. 6 .

An image processing device 11 according to the present embodiment is acomputer that causes a display 16 to display, as a three-dimensionalimage 53, three-dimensional data 52 representing a biological tissue 60.As shown in FIG. 4 , the image processing device 11 forms, in thethree-dimensional data 52, a cutting region 62 exposing a lumen 63 ofthe biological tissue 60 in the three-dimensional image 53. As shown inFIG. 2 , the image processing device 11 causes the display 16 todisplay, together with the three-dimensional image 53, a two-dimensionalimage 56 representing a cross section 64 of the biological tissue 60 anda region 65 corresponding to the cutting region 62 in the cross section64.

According to the present embodiment, it is possible to indicate how apart of a structure of the biological tissue 60 is cut out or omittedfrom an image. Therefore, a user can understand based on thetwo-dimensional image 56 the kind of structure corresponding to aportion of the biological tissue 60 that has been cut out or omitted,and which is not displayed in the three-dimensional image 53. Forexample, if the user is an operator, it is rather easy to perform atreatment for the inside of the biological tissue 60.

The image processing device 11 generates and updates thethree-dimensional data 52 based on tomographic data 51 of the biologicaltissue 60 acquired by a sensor that acquires the tomographic data 51while moving through the lumen 63 of the biological tissue 60. As shownin FIG. 2 , the image processing device 11 colors, in thethree-dimensional image 53, at least a voxel representing an innersurface 61 of the biological tissue 60 or a voxel that is adjacent tothe voxel representing the inner surface 61 and that represents thelumen 63 among a first voxel group 54 corresponding to the cross section64 indicated by the tomographic data 51 newly acquired by the sensor, ina manner of being distinguished from a second voxel group 55corresponding to another cross section of the biological tissue 60.

According to the present embodiment, it is possible to indicate whichpart in the three-dimensional image 53 the cross section 64 of thebiological tissue 60 indicated by the tomographic data 51 newly acquiredby the sensor corresponds to. Therefore, the user who observes the lumen63 of the biological tissue 60 using the three-dimensional image 53 canrather easily understand which part in the three-dimensional image 53information currently obtained by the sensor, that is, the latestinformation corresponds to.

In one modification of the present embodiment, the image processingdevice 11 may color not only the first voxel group 54 but also at leasta voxel representing the inner surface 61 or a voxel that is adjacent tothe voxel representing the inner surface 61 and that represents thelumen 63 among a voxel group corresponding to a cross section adjacentto the cross section 64 corresponding to the first voxel group 54, in amanner of being distinguished from a voxel group corresponding toanother cross section of the biological tissue 60. According to themodification, a width of the voxel group that is colored in a manner ofbeing distinguished from a voxel group corresponding to another crosssection in a sensor moving direction is widened, and the user can rathereasily recognize the voxel group in the three-dimensional image 53.

In one modification of the present embodiment, as shown in FIG. 14 , allvoxels representing the biological tissue 60 in the first voxel group 54may be colored in a manner of being distinguished from the second voxelgroup 55. According to the modification, since the first voxel group 54is colored in a manner of being distinguished from the second voxelgroup 55 on a cutting plane of the biological tissue 60 formed toobserve the lumen 63 of the biological tissue 60, it is relativelyeasier for the user to understand which part in the three-dimensionalimage 53 the latest information corresponds to.

In the present embodiment, the image processing device 11 causes thedisplay 16 to display the two-dimensional image 56 representing thecross section 64 together with the three-dimensional image 53 in whichat least the voxel representing the inner surface 61 of the biologicaltissue 60 or the voxel that is adjacent to the voxel representing theinner surface 61 and that represents the lumen 63 among the first voxelgroup 54 corresponding to the cross section 64 is colored in a manner ofbeing distinguished from the second voxel group 55 corresponding to theanother cross section. Therefore, a relation between the two-dimensionalimage 56 and the three-dimensional image 53 can be shown.

The biological tissue 60 includes, for example, a blood vessel or anorgan such as a heart. The biological tissue 60 is not limited to ananatomically single organ or a part of the anatomically single organ,and also includes a tissue having a lumen across a plurality of organs.Specific examples of such a tissue include a part of a vascular tissuefrom an upper portion of an inferior vena cava to a lower portion of asuperior vena cava through a right atrium. In the example in FIGS. 2 to4 , the biological tissue 60 is a blood vessel.

In FIG. 2 , an operation panel 81, the two-dimensional image 56, thethree-dimensional image 53, a first graphic element 86, and a secondgraphic element 87 are displayed on a screen 80.

The operation panel 81 is a graphical user interface (GUI) component forsetting the cutting region 62. The operation panel 81 can include acheck box 82 for selecting whether to activate the setting of thecutting region 62, a slider 83 for setting a base angle, a slider 84 forsetting an opening angle, and a check box 85 for selecting whether touse a centroid.

The base angle is a rotary angle of a straight line L1 of two straightlines L1 and L2 extending from one point M in a cross-sectional imagerepresenting the cross section 64 of the biological tissue 60.Therefore, setting the base angle corresponds to setting a direction ofthe straight line L1. The opening angle is an angle between the twostraight lines L1 and L2. Therefore, setting the opening anglecorresponds to setting an angle formed by the two straight lines L1 andL2. The point M is a centroid of the cross section 64. The point M maybe set at a point other than the centroid on the cross section 64 whenit is selected not to use the centroid.

The two-dimensional image 56 is an image obtained by processing across-sectional image. In the two-dimensional image 56, a color of theregion 65 corresponding to the cutting region 62 is changed to clearlyindicate which part of the cross section 64 is cut out (i.e., omitted)from the image 56.

In the present embodiment, a viewpoint when the three-dimensional image53 is displayed on the screen 80 is adjusted according to a position ofthe cutting region 62. The term “viewpoint” refers to a position of avirtual camera 71 disposed in a three-dimensional space. In thetwo-dimensional image 56, a position of the camera 71 is shown withrespect to the cross section 64.

In the present embodiment, the cutting region 62 can be determined usingthe two-dimensional image 56. Specifically, as shown in FIG. 3 , byadjusting the base angle or the opening angle and setting a position ora size of the region 65 divided by the two straight lines L1 and L2 inthe two-dimensional image 56, a position or a size of the cutting region62 can be set. For example, when the base angle is changed such that thestraight line L1 rotates counterclockwise by about 90 degrees, a region65 a after moving according to the change of the base angle is obtainedin a two-dimensional image 56 a. Then, the position of the cuttingregion 62 is adjusted according to a position of the region 65 a.Alternatively, when the opening angle is changed such that the anglebetween the two straight lines L1 and L2 is increased, a region 65 bafter enlargement according to the change in the opening angle isobtained in a two-dimensional image 56 b. Then, the size of the cuttingregion 62 is adjusted according to a size of the region 65 b. Byadjusting both the base angle and the opening angle and setting both theposition and the size of the region 65 in the two-dimensional image 56,both the position and the size of the cutting region 62 can also be set.The position of the camera 71 may be appropriately adjusted according tothe position or the size of the cutting region 62.

In the present embodiment, an image corresponding to a current positionof the sensor, that is, the latest image is always displayed as thetwo-dimensional image 56, but in one modification of the presentembodiment, an image corresponding to a position other than the currentposition of the sensor may be displayed as the two-dimensional image 56after the cutting region 62 is determined.

In one modification of the present embodiment, the base angle may be setby dragging the straight line L1 or by inputting a numerical value,instead of setting by operating the slider 83. Similarly, the openingangle may be set by dragging the straight line L2 or by inputting anumerical value, instead of setting by operating the slider 84.

In the three-dimensional image 53, the cutting region 62 determinedusing the two-dimensional image 56 is hidden or transparent. Inaddition, in the three-dimensional image 53, the color of the firstvoxel group 54 corresponding to the current position of the sensor ischanged in order to express a position in which the sensor is currentlypresent in a longitudinal direction of the lumen 63 and which iscurrently updated in real time.

In the present embodiment, as shown in FIG. 2 , the voxel representingthe inner surface 61 of the biological tissue 60 among the first voxelgroup 54 is colored in a manner of being distinguished from the secondvoxel group 55 by setting the color to be different from that of thesecond voxel group 55, but in the modification of the presentembodiment, as shown in FIG. 14 , all voxels representing the biologicaltissue 60 among the first voxel group 54 may be set to a differentcolor. In a further modification, instead of setting the first voxelgroup 54 and the second voxel group 55 to different colors, the firstvoxel group 54 may be colored in a manner of being distinguished fromthe second voxel group 55 by adjusting a contrast between the firstvoxel group 54 and the second voxel group 55.

The first graphic element 86 is a graphic element that represents amoving range of the sensor. The second graphic element 87 is a graphicelement representing a position of the sensor. In the presentembodiment, a combination of the first graphic element 86 and the secondgraphic element 87 is implemented as a slider. The first graphic element86 and the second graphic element 87 may be displayed at any position,and are displayed on the right of the three-dimensional image 53 in thepresent embodiment.

In FIG. 4 , an X-direction and a Y-direction orthogonal to theX-direction correspond to a lateral direction of the lumen 63 of thebiological tissue 60. A Z-direction orthogonal to the X-direction andthe Y-direction corresponds to the longitudinal direction of the lumen63 of the biological tissue 60.

In an example in FIG. 4 , the check box 85 on the operation panel 81 isin a checked state, that is, it is selected to use the centroid. Theimage processing device 11 calculates positions of centroids B1, B2, B3,and B4 of cross sections C1, C2, C3, and C4 of the biological tissue 60using the three-dimensional data 52. The image processing device 11sets, as cutting planes P1 and P2, two planes that intersect at a singleline Lb passing through the positions of the centroids B1, B2, B3, andB4 and that include the respective two straight lines L1 and L2. Forexample, when the point M shown in FIG. 2 is the point B3, the straightline L1 is an intersection line between the cross section C3 and thecutting plane P1, and the straight line L2 is an intersection linebetween the cross section C3 and the cutting plane P2. The imageprocessing device 11 forms, as the cutting region 62 in thethree-dimensional data 52, a region interposed between the cuttingplanes P1 and P2 in the three-dimensional image 53 and exposing thelumen 63 of the biological tissue 60.

In the case of a three-dimensional model of the bent blood vessel asshown in FIG. 4 , when the three-dimensional model is cut with one planeto display the lumen 63, there is a case in which the inside of theblood vessel cannot be correctly displayed. In the present embodiment,as shown in FIG. 4 , by continuously capturing centroids of the bloodvessel, the three-dimensional model can be cut such that the inside ofthe blood vessel can be reliably displayed.

In FIG. 4 , for convenience, the four cross sections C1, C2, C3, and C4are illustrated as a plurality of lateral cross sections of the lumen 63of the biological tissue 60, but the number of cross sections serving ascalculation targets of the centroid positions is not limited to four,and is preferably the same as the number of cross-sectional imagesacquired by IVUS.

In an example different from that in FIG. 4 , the check box 85 on theoperation panel 81 is in a not-checked state, that is, it is selectednot to use the centroid. In such an example, the image processing device11 sets, as the cutting planes P1 and P2, two planes that intersect atany single line passing through the point M, such as a straight linepassing through the point M and extending in the Z-direction, and thatinclude the respective two straight lines L1 and L2.

A configuration of an image processing system 10 according to thepresent embodiment will be described with reference to FIG. 1 .

The image processing system 10 can include the image processing device11, a cable 12, a drive unit 13, a keyboard 14, a mouse 15, and thedisplay 16.

The image processing device 11 is a dedicated computer specialized forimage diagnosis in the present embodiment, and may also be ageneral-purpose computer such as a personal computer (PC).

The cable 12 is used to connect the image processing device 11 and thedrive unit 13.

The drive unit 13 is a device to be used by being connected to a probe20 shown in FIG. 5 to drive the probe 20. The drive unit 13 is alsoreferred to as a motor drive unit (MDU). The probe 20 is applied toIVUS. The probe 20 is also referred to as an IVUS catheter or an imagediagnostic catheter.

The keyboard 14, the mouse 15, and the display 16 are connected to theimage processing device 11 via a cable or wirelessly. The display 16 canbe, for example, a liquid crystal display (LCD), an organic electroluminescence (EL) display, or a head-mounted display (HMD).

The image processing system 10 optionally further includes a connectionterminal 17 and a cart unit 18.

The connection terminal 17 is used to connect the image processingdevice 11 and an external device. The connection terminal 17 can be, forexample, a universal serial bus (USB) terminal. The external device canbe, for example, a recording medium such as a magnetic disc drive, amagneto-optical disc drive, or an optical disc drive.

The cart unit 18 can be a cart equipped with casters for movement. Theimage processing device 11, the cable 12, and the drive unit 13 aredisposed on a cart body of the cart unit 18. The keyboard 14, the mouse15, and the display 16 can be disposed on the uppermost table of thecart unit 18.

Configurations of the probe 20 and the drive unit 13 according to thepresent embodiment will be described with reference to FIG. 5 .

The probe 20 can include a drive shaft 21, a hub 22, a sheath 23, anouter tube 24, an ultrasound transducer 25, and a relay connector 26.

The drive shaft 21 passes through the sheath 23 to be inserted into alumen in a living body and the outer tube 24 connected to a proximal endof the sheath 23, and extends to an inside of the hub 22 provided at aproximal end of the probe 20. The drive shaft 21 is provided with theultrasound transducer 25, which transmits and receives signals, at adistal end of the drive shaft 21, and is rotatably provided in thesheath 23 and the outer tube 24. The relay connector 26 connects thesheath 23 and the outer tube 24.

The hub 22, the drive shaft 21, and the ultrasound transducer 25 areconnected to each other to integrally move forward and backward in anaxial direction. Therefore, for example, when the hub 22 is pressedtoward a distal side, the drive shaft 21 and the ultrasound transducer25 move inside the sheath 23 toward the distal side. For example, whenthe hub 22 is pulled toward a proximal side, the drive shaft 21 and theultrasound transducer 25 move inside the sheath 23 toward the proximalside as indicated by an arrow.

The drive unit 13 can include a scanner unit 31, a slide unit 32, and abottom cover 33.

The scanner unit 31 is connected to the image processing device 11 viathe cable 12. The scanner unit 31 can include a probe connection section34 connected to the probe 20, and a scanner motor 35. The scanner motor35 can be a drive source for rotating the drive shaft 21.

The probe connection section 34 is freely detachably connected to theprobe 20 through an insertion port 36 of the hub 22 provided at theproximal end of the probe 20. Inside the hub 22, a proximal end of thedrive shaft 21 is rotatably supported, and a rotational force of thescanner motor 35 is transmitted to the drive shaft 21. A signal istransmitted and received between the drive shaft 21 and the imageprocessing device 11 via the cable 12. In the image processing device11, generation of a tomographic image of a body lumen and imageprocessing are executed based on the signal transmitted from the driveshaft 21.

The slide unit 32 is mounted with the scanner unit 31 in a manner ofbeing capable of moving forward and backward, and is mechanically andelectrically connected to the scanner unit 31. The slide unit 32 caninclude a probe clamp section 37, a slide motor 38, and a switch group39.

The probe clamp section 37 is disposed coaxially with the probeconnection section 34 on a distal side relative to the probe connectionsection 34, and supports the probe 20 to be connected to the probeconnection section 34.

The slide motor 38 is a drive source that generates a drive force in theaxial direction. The scanner unit 31 moves forward and backward whendriven by the slide motor 38, and the drive shaft 21 moves forward andbackward in the axial direction accordingly. The slide motor 38 can be,for example, a servo motor.

The switch group 39 can include, for example, a forward switch and apull-back switch that are pressed when the scanner unit 31 is to bemoved forward or backward, and a scan switch that is pressed when imagedrawing is to be started or ended. Various switches may be included inthe switch group 39 as necessary without being limited to the examplehere.

When the forward switch is pressed, the slide motor 38 rotates forward,and the scanner unit 31 moves forward. Meanwhile, when the pull-backswitch is pressed, the slide motor 38 rotates backward, and the scannerunit 31 moves backward.

When the scan switch is pressed, the image drawing is started, thescanner motor 35 is driven, and the slide motor 38 is driven to move thescanner unit 31 backward. The user such as the operator connects theprobe 20 to the scanner unit 31 in advance, and the drive shaft 21rotates and moves toward the proximal side in the axial direction uponthe start of the image drawing. When the scan switch is pressed again,the scanner motor 35 and the slide motor 38 are stopped, and the imagedrawing is ended.

The bottom cover 33 covers a bottom and an entire circumference of aside surface on a bottom side of the slide unit 32, and is capable ofmoving toward and away from the bottom of the slide unit 32.

A configuration of the image processing device 11 will be described withreference to FIG. 6 .

The image processing device 11 includes a control unit 41, a storageunit 42, a communication unit 43, an input unit 44, and an output unit45.

The control unit 41 includes at least one processor, at least oneprogrammable circuit, at least one dedicated circuit, or any combinationof the at least one processor, the at least one programmable circuit,and the at least one dedicated circuit. The processor is ageneral-purpose processor such as a central processing unit (CPU) or agraphics processing unit (GPU), or a dedicated processor specialized forspecific processing. The programmable circuit can be, for example, afield-programmable gate array (FPGA). The dedicated circuit can be, forexample, an application specific integrated circuit (ASIC). The controlunit 41 executes processing related to an operation of the imageprocessing device 11 while controlling each unit of the image processingsystem 10 including the image processing device 11.

The storage unit 42 includes at least one semiconductor memory, at leastone magnetic memory, at least one optical memory, or any combination ofthe at least one semiconductor memory, the at least one magnetic memory,and the at least one optical memory. The semiconductor memory can be,for example, a random access memory (RAM) or a read only memory (ROM).The RAM can be, for example, a static random access memory (SRAM) or adynamic random access memory (DRAM). The ROM can be, for example, anelectrically erasable programmable read only memory (EEPROM). Thestorage unit 42 functions as, for example, a main storage device, anauxiliary storage device, or a cache memory. The storage unit 42 storesdata used for the operation of the image processing device 11, such asthe tomographic data 51, and data obtained by the operation of the imageprocessing device 11, such as the three-dimensional data 52 and thethree-dimensional image 53.

The communication unit 43 includes at least one communication interface.The communication interface is, for example, a wired local area network(LAN) interface, a wireless LAN interface, or an image diagnosticinterface for receiving IVUS signals and performing analog to digital(A/D) conversion for the IVUS signals. The communication unit 43receives the data used for the operation of the image processing device11 and transmits the data obtained by the operation of the imageprocessing device 11. In the present embodiment, the drive unit 13 isconnected to the image diagnostic interface included in thecommunication unit 43.

The input unit 44 includes at least one input interface. The inputinterface can be, for example, a USB interface, a High-DefinitionMultimedia Interface (HDMI®) interface, or an interface compatible witha short-range wireless communication standard such as Bluetooth®. Theinput unit 44 receives an operation by the user such as an operation ofinputting data used for the operation of the image processing device 11.In the present embodiment, the keyboard 14 and the mouse 15 areconnected to the USB interface or the interface compatible withshort-range wireless communication included in the input unit 44. When atouch screen is provided integrally with the display 16, the display 16may be connected to the USB interface or the HDMI interface included inthe input unit 44.

The output unit 45 includes at least one output interface. The outputinterface can be, for example, a USB interface, an HDMI interface, or aninterface compatible with a short-range wireless communication standardsuch as Bluetooth. The output unit 45 outputs the data obtained by theoperation of the image processing device 11. In the present embodiment,the display 16 is connected to the USB interface or the HDMI interfaceincluded in the output unit 45.

A function of the image processing device 11 is implemented by executingan image processing program according to the present embodiment by aprocessor as the control unit 41. That is, the function of the imageprocessing device 11 is implemented by software. The image processingprogram causes a computer to function as the image processing device 11by causing the computer to execute the operation of the image processingdevice 11. That is, the computer functions as the image processingdevice 11 by executing the operation of the image processing device 11according to the image processing program.

The program may be stored in a non-transitory computer-readable mediumin advance. The non-transitory computer-readable medium can be, forexample, a flash memory, a magnetic recording device, an optical disc, amagneto-optical recording medium, or a ROM. Distribution of the programis executed by, for example, selling, transferring, or lending aportable medium such as a secure digital (SD) card, a digital versatiledisc (DVD) or a compact disc read only memory (CD-ROM) storing theprogram. The program may be distributed by storing the program in astorage of a server in advance and transferring the program from theserver to another computer. The program may be provided as a programproduct.

For example, the computer temporarily stores, in the main storagedevice, the program stored in the portable medium or the programtransferred from the server. The computer reads, by the processor, theprogram stored in the main storage device, and executes, by theprocessor, processing according to the read program. The computer mayread the program directly from the portable medium and execute theprocessing according to the program. Each time the program istransferred from the server to the computer, the computer maysequentially execute processing according to the received program. Theprocessing may be executed by a so-called application service provider(ASP) type service in which the function is implemented only byexecution instruction and result acquisition without transferring theprogram from the server to the computer. The program includesinformation provided for processing by an electronic computer andconforming to the program. For example, data that is not a directcommand to the computer but has a property of defining the processing ofthe computer corresponds to the “information conforming to the program”.

The functions of the image processing device 11 may be partially orentirely implemented by the programmable circuit or the dedicatedcircuit as the control unit 41. That is, the functions of the imageprocessing device 11 may be partially or entirely implemented byhardware.

An operation of the image processing system 10 according to the presentembodiment will be described with reference to FIGS. 7 and 8 . Theoperation of the image processing system 10 corresponds to an imagedisplay method according to the present embodiment.

Before a start of a flow in FIG. 7 , the probe 20 is primed by the user.Thereafter, the probe 20 is fitted into the probe connection section 34and the probe clamp section 37 of the drive unit 13, and is connectedand fixed to the drive unit 13. Then, the probe 20 is inserted to atarget site in the biological tissue 60 such as a blood vessel or aheart.

In S101, the scan switch included in the switch group 39 is pressed, anda so-called pull-back operation is executed by pressing the pull-backswitch included in the switch group 39. The probe 20 transmits anultrasound inside the biological tissue 60 by the ultrasound transducer25 that moves backward in the axial direction by the pull-backoperation. The ultrasound transducer 25 radially transmits theultrasound while moving inside the biological tissue 60. The ultrasoundtransducer 25 receives a reflected wave of the transmitted ultrasound.The probe 20 inputs a signal of the reflected wave received by theultrasound transducer 25 to the image processing device 11. The controlunit 41 of the image processing device 11 processes the input signal tosequentially generate cross-sectional images of the biological tissue60, thereby acquiring the tomographic data 51, which includes aplurality of cross-sectional images.

Specifically, the probe 20 transmits, by the ultrasound transducer 25,the ultrasound in a plurality of directions from a rotation center to anoutside while causing the ultrasound transducer 25 to rotate in acircumferential direction and to move in the axial direction inside thebiological tissue 60. The probe 20 receives, by the ultrasoundtransducer 25, the reflected wave from a reflecting object present ineach of the plurality of directions inside the biological tissue 60. Theprobe 20 transmits the signal of the received reflected wave to theimage processing device 11 via the drive unit 13 and the cable 12. Thecommunication unit 43 of the image processing device 11 receives thesignal transmitted from the probe 20. The communication unit 43 performsA/D conversion for the received signal. The communication unit 43 inputsthe signal after A/D conversion to the control unit 41. The control unit41 processes the input signal to calculate an intensity valuedistribution of the reflected wave from the reflecting object present ina transmission direction of the ultrasound of the ultrasound transducer25. The control unit 41 sequentially generates two-dimensional imageshaving a luminance value distribution corresponding to the calculatedintensity value distribution as the cross-sectional images of thebiological tissue 60, thereby acquiring the tomographic data 51 which isa data set of the cross-sectional images. The control unit 41 stores theacquired tomographic data 51 in the storage unit 42.

In the present embodiment, the signal of the reflected wave received bythe ultrasound transducer 25 corresponds to raw data of the tomographicdata 51, and the cross-sectional images generated by processing thesignal of the reflected wave with the image processing device 11correspond to processed data of the tomographic data 51.

In one modification of the present embodiment, the control unit 41 ofthe image processing device 11 may store the signal input from the probe20 as it is in the storage unit 42 as the tomographic data 51.Alternatively, the control unit 41 may store data indicating theintensity value distribution of the reflected wave calculated byprocessing the signal input from the probe 20 in the storage unit 42 asthe tomographic data 51. That is, the tomographic data 51 is not limitedto the data set of the cross-sectional images of the biological tissue60, and may be data representing a cross section of the biologicaltissue 60 at each moving position of the ultrasound transducer 25 in anyformat.

In one modification of the present embodiment, an ultrasound transducerthat transmits the ultrasound in the plurality of directions withoutrotation may be used instead of the ultrasound transducer 25 thattransmits the ultrasound in the plurality of directions while rotatingin the circumferential direction.

In one modification of the present embodiment, the tomographic data 51may be acquired by using optical frequency domain imaging (OFDI) oroptical coherence tomography (OCT) instead of being acquired by usingIVUS. When OFDI or OCT is used, as a sensor that acquires thetomographic data 51 while moving in the lumen 63 of the biologicaltissue 60, a sensor that acquires the tomographic data 51 by emittinglight in the lumen 63 of the biological tissue 60 is used instead of theultrasound transducer 25 that acquires the tomographic data 51 bytransmitting the ultrasound in the lumen 63 of the biological tissue 60.

In one modification of the present embodiment, instead of the imageprocessing device 11 generating the data set of the cross-sectionalimages of the biological tissue 60, another device may generate the samedata set, and the image processing device 11 may acquire the data setfrom another device. That is, instead of the control unit 41 of theimage processing device 11 processing the IVUS signal to generate thecross-sectional images of the biological tissue 60, another device mayprocess the IVUS signal to generate the cross-sectional images of thebiological tissue 60 and input the generated cross-sectional images tothe image processing device 11.

In S102, the control unit 41 of the image processing device 11 generatesthe three-dimensional data 52 of the biological tissue 60 based on thetomographic data 51 acquired in S101. That is, the control unit 41generates the three-dimensional data 52 based on the tomographic data 51acquired by the sensor. Here, when the already generatedthree-dimensional data 52 is present, it is preferable to update onlydata at a location corresponding to the updated tomographic data 51,instead of regenerating all the three-dimensional data 52 from thebeginning. In this case, a data processing amount when generating thethree-dimensional data 52 can be reduced, and a real-time property ofthe three-dimensional image 53 in S103 can be improved, and wherein S103is subsequent or after S102.

Specifically, the control unit 41 of the image processing device 11generates the three-dimensional data 52 of the biological tissue 60 bystacking the cross-sectional images of the biological tissue 60 includedin the tomographic data 51 stored in the storage unit 42, and convertingthe same into three-dimensional data. As a method for three-dimensionalconversion, any method among a rendering method such as surfacerendering or volume rendering, and various types of processing such astexture mapping including environment mapping, and bump mapping, whichare associated with the rendering method, can be used. The control unit41 stores the generated three-dimensional data 52 in the storage unit42.

In S103, the control unit 41 of the image processing device 11 causesthe display 16 to display the three-dimensional data 52 generated inS102 as the three-dimensional image 53. At this time, the control unit41 may set an angle for displaying the three-dimensional image 53 to anyangle. The control unit 41 causes the display 16 to display the latestcross-sectional image included in the tomographic data 51 acquired inS101 together with the three-dimensional image 53.

Specifically, the control unit 41 of the image processing device 11generates the three-dimensional image 53 based on the three-dimensionaldata 52 stored in the storage unit 42. The control unit 41 causes thedisplay 16 to display, via the output unit 45, the generatedthree-dimensional image 53 and the latest cross-sectional image amongthe cross-sectional images of the biological tissue 60 included in thetomographic data 51 stored in the storage unit 42.

In the present embodiment, the control unit 41 of the image processingdevice 11 colors, in the three-dimensional image 53, the voxelrepresenting the inner surface 61 of the biological tissue 60 among thefirst voxel group 54 corresponding to the cross section 64 indicated bythe tomographic data 51 newly acquired by the sensor, in a manner ofbeing distinguished from the second voxel group 55 corresponding to theanother cross section of the biological tissue 60. Specifically, asshown in FIG. 2 , the control unit 41 sets the color of the voxelrepresenting the inner surface 61 of the biological tissue 60 among thefirst voxel group 54 to a color different from any color of the secondvoxel group 55, thereby coloring the voxel representing the innersurface 61 of the biological tissue 60 among the first voxel group 54 ina manner of being distinguished from the second voxel group 55.

In one modification of the present embodiment, as shown in FIG. 14 , thecontrol unit 41 of the image processing device 11 may color all of thevoxels representing the biological tissue 60 among the first voxel group54 in a manner of being distinguished from the second voxel group 55.Specifically, the control unit 41 may set the color of all of the voxelsrepresenting the biological tissue 60 among the first voxel group 54 toa color different from any color of the second voxel group 55, therebycoloring all of the voxels representing the biological tissue 60 amongthe first voxel group 54 in a manner of being distinguished from thesecond voxel group 55.

In the present embodiment, the control unit 41 of the image processingdevice 11 combines the first graphic element 86 and the second graphicelement 87 and causes the display 16 to display the same together withthe three-dimensional image 53. Specifically, as shown in FIG. 2 , thecontrol unit 41 causes the slider implemented by combining the firstgraphic element 86 and the second graphic element 87 to be displayed onthe right of the three-dimensional image 53 via the output unit 45.

In the present embodiment, the control unit 41 of the image processingdevice 11 causes the display 16 to display the first graphic element 86in a direction in which the longitudinal direction of the lumen 63 inthe three-dimensional image 53 and a long axis direction of the firstgraphic element 86 are parallel to each other. Specifically, as shown inFIG. 2 , the control unit 41 matches the moving range of the sensorindicated by the first graphic element 86 with a display range of thethree-dimensional image 53 in a vertical direction of the screen 80, andmatches the position of the sensor indicated by the second graphicelement 87 with a position of the first voxel group 54.

In S104, if there is an operation of setting the angle for displayingthe three-dimensional image 53 as a change operation by the user,processing in S105 is executed. If there is no change operation by theuser, processing in S106 is executed.

In S105, the control unit 41 of the image processing device 11 receives,via the input unit 44, the operation of setting the angle for displayingthe three-dimensional image 53. The control unit 41 adjusts the anglefor displaying the three-dimensional image 53 to the set angle. Then, inS103, the control unit 41 causes the display 16 to display thethree-dimensional image 53 at the angle set in S105.

Specifically, the control unit 41 of the image processing device 11receives, via the input unit 44, an operation by the user of rotatingthe three-dimensional image 53 displayed on the display 16 by using thekeyboard 14, the mouse 15, or the touch screen provided integrally withthe display 16. The control unit 41 interactively adjusts the angle fordisplaying the three-dimensional image 53 on the display 16 according tothe operation by the user. Alternatively, the control unit 41 receives,via the input unit 44, an operation by the user of inputting a numericalvalue of the angle for displaying the three-dimensional image 53 byusing the keyboard 14, the mouse 15, or the touch screen providedintegrally with the display 16. The control unit 41 adjusts the anglefor displaying the three-dimensional image 53 on the display 16 inaccordance with the input numerical value.

In S106, if the tomographic data 51 is updated, processing in S107 andS108 is executed. If the tomographic data 51 is not updated, thepresence or absence of the change operation by the user is confirmedagain in S104.

In S107, similar to the processing in S101, the control unit 41 of theimage processing device 11 processes the signal input from the probe 20to newly generate cross-sectional images of the biological tissue 60,thereby acquiring the tomographic data 51 including at least one newcross-sectional image.

In S108, the control unit 41 of the image processing device 11 updatesthe three-dimensional data 52 of the biological tissue 60 based on thetomographic data 51 acquired in S107. That is, the control unit 41updates the three-dimensional data 52 based on the tomographic data 51acquired by the sensor. Then, in S103, the control unit 41 causes thedisplay 16 to display the three-dimensional data 52 updated in S108 asthe three-dimensional image 53. The control unit 41 causes the display16 to display the latest cross-sectional image included in thetomographic data 51 acquired in S107 together with the three-dimensionalimage 53. In S108, it is preferable to update only data at a locationcorresponding to the updated tomographic data 51. In this case, the dataprocessing amount when generating the three-dimensional data 52 can bereduced, and the real-time property of the three-dimensional image 53can be improved in S108.

In S111, if there is an operation of setting the cutting region 62 as asetting operation by the user, processing in S112 is executed.

In S112, the control unit 41 of the image processing device 11 receives,via the input unit 44, the operation of setting the cutting region 62.

Specifically, the control unit 41 of the image processing device 11receives, via the input unit 44, an operation of setting the region 65corresponding to the cutting region 62 on the cross-sectional imagedisplayed on the display 16 in S103. In the present embodiment, thecontrol unit 41 receives an operation of setting the two straight linesL1 and L2 extending from the point M in the cross-sectional image as theoperation of setting the region 65 corresponding to the cutting region62.

More specifically, the control unit 41 of the image processing device 11receives, via the input unit 44, an operation by the user of designatingthe base angle and the opening angle by using the keyboard 14, the mouse15, or the touch screen provided integrally with the display 16 on theoperation panel 81 as shown in FIG. 2 . That is, the control unit 41receives an operation of designating the direction of the straight lineL1 of the two straight lines L1 and L2 and the angle formed by the twostraight lines L1 and L2 as the operation of setting the two straightlines L1 and L2. Here, the check box 85 on the operation panel 81 is ina checked state, that is, it is selected to use the centroid.

In one modification of the present embodiment, the control unit 41 ofthe image processing device 11 may receive, via the input unit 44, anoperation by the user of drawing the two straight lines L1 and L2 byusing the keyboard 14, the mouse 15, or the touch screen providedintegrally with the display 16 on the cross-sectional image displayed onthe display 16. That is, the control unit 41 may receive the operationof drawing the two straight lines L1 and L2 on the cross-sectional imageas the operation of setting the two straight lines L1 and L2.

In S113, the control unit 41 of the image processing device 11calculates centroid positions of a plurality of lateral cross sectionsof the lumen 63 of the biological tissue 60 by using the latestthree-dimensional data 52 stored in the storage unit 42. The latestthree-dimensional data 52 is the three-dimensional data 52 generated inS102 if the processing in S108 is not executed, and is thethree-dimensional data 52 updated in S108 if the processing in S108 isexecuted. Here, when the already generated three-dimensional data 52 ispresent, it is preferable to update only data at a locationcorresponding to the updated tomographic data 51, instead ofregenerating all the three-dimensional data 52 from the beginning. Inthis case, the data processing amount when generating thethree-dimensional data 52 can be reduced, and the real-time property ofthe three-dimensional image 53 in a subsequent S117 can be improved.

Specifically, as shown in FIG. 9 , if the control unit 41 of the imageprocessing device 11 generates a corresponding new cross-sectional imagein S107 for each of the plurality of cross-sectional images generated inS101, the control unit 41 replaces each of the plurality ofcross-sectional images generated in S101 with the new cross-sectionalimage, and then binarizes the cross-sectional image. As shown in FIG. 10, the control unit 41 extracts a point group on an inner surface of thebiological tissue 60 from the binarized cross-sectional image. Forexample, the control unit 41 extracts a point group on an inner surfaceof a blood vessel by extracting, one by one, points corresponding to aninner surface of a main blood vessel along a vertical direction of thecross-sectional image having an r-axis as a horizontal axis and a θ-axisas a vertical axis. The control unit 41 may simply obtain a centroid ofthe extracted point group on the inner surface, but in this case, sincethe point group is not uniformly sampled over the inner surface, acentroid position shifts. Therefore, in the present embodiment, thecontrol unit 41 calculates a convex hull of the extracted point group onthe inner surface, and calculates a centroid position C_(n) = (C_(x),C_(y)) by using a formula for obtaining a centroid of a polygon asfollows. In the following formula, n vertices (x₀, y₀), (x₁, y₁), ...,(x_(n-1), y_(n-1)) are regarded as being present on the convex hullcounterclockwise as the point group on the inner surface as shown inFIG. 10 , and (x_(n), y_(n)) is regarded as (x₀, y₀).

$\begin{array}{l}{C_{x} = \frac{1}{6A}{\sum_{i = 0}^{n - 1}{\left( {x_{i} + x_{i + 1}} \right)\left( {x_{i}y_{i + 1} - x_{i + 1}y_{i}} \right)}}} \\{C_{y} = \frac{1}{6A}{\sum_{i = 0}^{n - 1}{\left( {y_{i} + y_{i + 1}} \right)\left( {x_{i}y_{i + 1} - x_{i + 1}y_{i}} \right)}}} \\{A = \frac{1}{2}{\sum_{i = 0}^{N - 1}\left( {x_{i}y_{i + 1} - x_{i + 1}y_{i}} \right)}}\end{array}$

The centroid positions obtained as results are shown in FIG. 11 . InFIG. 11 , a point Cn is a center of the cross-sectional image. A pointBp is a centroid of the point group on the inner surface. A point Bv isa centroid of vertices of the polygon. A point Bx is a centroid of thepolygon serving as the convex hull.

As a method of calculating the centroid position of the blood vessel, amethod other than the method of calculating the centroid position of thepolygon serving as the convex hull may be used. For example, withrespect to an original cross-sectional image that is not binarized, amethod of calculating a center position of a maximum circle that fallswithin the main blood vessel as the centroid position may be used.Alternatively, with respect to the binarized cross-sectional imagehaving the r-axis as the horizontal axis and the θ-axis as the verticalaxis, a method of calculating an average position of pixels in a mainblood vessel region as the centroid position may be used. The samemethod as described above may also be used when the biological tissue 60is not a blood vessel.

In S114, the control unit 41 of the image processing device 11 smoothscalculation results of the centroid positions in S113.

As shown in FIG. 12 , when the calculation results of the centroidpositions are viewed as a time function, it can be seen that aninfluence of pulsation is large. Therefore, in the present embodiment,the control unit 41 of the image processing device 11 smooths thecalculation results of the centroid positions by using moving average asindicated by a broken line in FIG. 13 .

As a smoothing method, a method other than moving average may be used.For example, an exponential smoothing method, a kernel method, localregression, a Ramer-Douglas-Peucker algorithm, a Savitzky-Golay method,smoothing spline, or stretched grid method (SGM) may be used.Alternatively, a method of executing a fast Fourier transform and thenremoving a high frequency component may be used. Alternatively, a Kalmanfilter or a low-pass filter such as a Butterworth filter, a Chebyshevfilter, a digital filter, an elliptic filter, or a Kolmogorov-Zurbenko(KZ) filter may be used.

When smoothing is simply executed, the centroid positions may enter thetissue. In this case, the control unit 41 may divide the calculationresults of the centroid positions according to positions of a pluralityof lateral cross sections of the lumen 63 of the biological tissue 60 inthe longitudinal direction of the lumen 63 of the biological tissue 60,and may smooth each of the divided calculation results. That is, when acurve of the centroid positions as indicated by the broken line in FIG.13 overlaps a tissue region, the control unit 41 may divide the curve ofthe centroid positions into a plurality of sections and smooth eachsection. Alternatively, the control unit 41 may adjust a degree ofsmoothing to be executed on the calculation results of the centroidpositions according to the positions of the plurality of lateral crosssections of the lumen 63 of the biological tissue 60 in the longitudinaldirection of the lumen 63 of the biological tissue 60. That is, when thecurve of the centroid positions as indicated by the broken line in FIG.13 overlaps the tissue region, the control unit 41 may decrease thedegree of smoothing to be executed on a part of a section includingoverlapping points.

In S115, as shown in FIG. 4 , the control unit 41 of the imageprocessing device 11 sets, as the cutting planes P1 and P2, two planesintersecting at the single line Lb passing through the centroidpositions calculated in S113. In the present embodiment, the controlunit 41 smooths the calculation results of the centroid positions inS114 and then sets the cutting planes P1 and P2, but the processing inS114 may be omitted.

Specifically, the control unit 41 of the image processing device 11sets, as the line Lb, a curve of the centroid positions obtained as aresult of the smoothing in S114. The control unit 41 sets, as thecutting planes P1 and P2, the two planes that intersect at the set lineLb and that include the respective two straight lines L1 and L2 set inS112. The control unit 41 specifies three-dimensional coordinatesintersecting with the cutting planes P1 and P2 of the biological tissue60 in the latest three-dimensional data 52 stored in the storage unit 42as three-dimensional coordinates of an edge of an opening exposing thelumen 63 of the biological tissue 60 in the three-dimensional image 53.The control unit 41 stores the specified three-dimensional coordinatesin the storage unit 42.

In S116, the control unit 41 of the image processing device 11 forms, asthe cutting region 62 in the three-dimensional data 52, the regioninterposed between the cutting planes P1 and P2 in the three-dimensionalimage 53 and exposing the lumen 63 of the biological tissue 60.

Specifically, the control unit 41 of the image processing device 11 setsa portion in the latest three-dimensional data 52 stored in the storageunit 42 that is specified by the three-dimensional coordinates stored inthe storage unit 42 to be hidden or to be transparent when thethree-dimensional image 53 is to be displayed on the display 16. Thatis, the control unit 41 forms the cutting region 62 in accordance withthe region 65 set in S112.

In S117, the control unit 41 of the image processing device 11 causesthe display 16 to display the three-dimensional data 52 having thecutting region 62 formed in S116 as the three-dimensional image 53. Thecontrol unit 41 causes the display 16 to display, together with thethree-dimensional image 53, the two-dimensional image 56 representingthe cross section 64 indicated by the tomographic data 51 newly acquiredby the sensor and the region 65 corresponding to the cutting region 62in the cross section 64, which are represented by the cross-sectionalimage displayed on the display 16 in S103.

Specifically, the control unit 41 of the image processing device 11processes the latest cross-sectional image among the cross-sectionalimages of the biological tissue 60 included in the tomographic data 51stored in the storage unit 42 to generate the two-dimensional image 56as shown in FIG. 2 . The control unit 41 generates the three-dimensionalimage 53 as shown in FIG. 2 in which the portion specified by thethree-dimensional coordinates stored in the storage unit 42 is hidden ortransparent. The control unit 41 causes the display 16 to display thegenerated two-dimensional image 56 and three-dimensional image 53 viathe output unit 45.

In the present embodiment, as shown in FIG. 2 , the control unit 41 ofthe image processing device 11 generates, as the two-dimensional image56, an image in which the color of the region 65 corresponding to thecutting region 62 is represented by a color different from that of aremaining region. For example, a white portion in a general IVUS imagemay be changed to red in the region 65.

In S118, if there is an operation of setting the cutting region 62 asthe change operation by the user, processing in S119 is executed. Ifthere is no change operation by the user, processing in S120 isexecuted.

In S119, similar to the processing in S112, the control unit 41 of theimage processing device 11 receives, via the input unit 44, theoperation of setting the cutting region 62. Then, the processing in S115and the subsequent steps is executed.

In S120, if the tomographic data 51 is updated, processing in S121 andS122 is executed. If the tomographic data 51 is not updated, thepresence or absence of the change operation by the user is confirmedagain in S118.

In S121, similar to the processing in S101 or S107, the control unit 41of the image processing device 11 processes the signal input from theprobe 20 to newly generate cross-sectional images of the biologicaltissue 60, thereby acquiring the tomographic data 51 including at leastone new cross-sectional image.

In S122, the control unit 41 of the image processing device 11 updatesthe three-dimensional data 52 of the biological tissue 60 based on thetomographic data 51 acquired in S121. Thereafter, the processing in S113and the subsequent steps is executed. In S122, it is preferable toupdate only data at a location corresponding to the updated tomographicdata 51. In this case, the data processing amount when generating thethree-dimensional data 52 can be reduced, and the real-time property ofdata processing in S113 and the subsequent steps can be improved.

As described above, in the present embodiment, the control unit 41 ofthe image processing device 11 causes the display 16 to display, as thethree-dimensional image 53, the three-dimensional data 52 representingthe biological tissue 60. The control unit 41 forms, in thethree-dimensional data 52, the cutting region 62 exposing the lumen 63of the biological tissue 60 in the three-dimensional image 53. Thecontrol unit 41 causes the display 16 to display, together with thethree-dimensional image 53, the two-dimensional image 56 representingthe cross section 64 of the biological tissue 60 and the region 65corresponding to the cutting region 62 in the cross section 64.

According to the present embodiment, it is possible to indicate how apart of the structure of the biological tissue 60 is cut out. Therefore,the user can understand based on the two-dimensional image 56 the kindof structure of a portion of the biological tissue 60 that is cut out oromitted and not displayed in the three-dimensional image 53. Forexample, if the user is an operator, it is relatively easy to perform atreatment for the inside of the biological tissue 60.

In the present embodiment, the control unit 41 of the image processingdevice 11 generates and updates the three-dimensional data 52representing the biological tissue 60 based on the tomographic data 51of the biological tissue 60 acquired by the sensor that acquires thetomographic data 51 while moving through the lumen 63 of the biologicaltissue 60. The control unit 41 causes the display 16 to display thethree-dimensional data 52 as the three-dimensional image 53. The controlunit 41 colors, in the three-dimensional image 53, at least the voxelrepresenting the inner surface 61 of the biological tissue 60 or thevoxel that is adjacent to the voxel representing the inner surface 61and that represents the lumen 63 among the first voxel group 54corresponding to the cross section 64 indicated by the tomographic data51 newly acquired by the sensor, in a manner of being distinguished fromthe second voxel group 55 corresponding to the another cross section ofthe biological tissue 60.

According to the present embodiment, it is possible to indicate whichpart in the three-dimensional image 53 the cross section 64 of thebiological tissue 60 indicated by the tomographic data 51 newly acquiredby the sensor corresponds to. Therefore, the user who observes the lumen63 of the biological tissue 60 using the three-dimensional image 53 canrelatively easily understand which part in the three-dimensional image53 the information currently obtained by the sensor, that is, the latestinformation corresponds to.

The present disclosure is not limited to the above-describedembodiments. For example, two or more blocks described in a blockdiagram may be integrated, or one block may be divided. Instead ofexecuting two or more steps described in the flowchart in time seriesaccording to the description, the steps may be executed in parallel orin a different order according to a processing capability of the devicethat executes each step or as necessary. In addition, modifications canbe made without departing from a gist of the present disclosure.

The detailed description above describes embodiments of an imageprocessing device, an image processing system, an image display method,and an image processing program. The invention is not limited, however,to the precise embodiments and variations described. Various changes,modifications and equivalents may occur to one skilled in the artwithout departing from the spirit and scope of the invention as definedin the accompanying claims. It is expressly intended that all suchchanges, modifications and equivalents which fall within the scope ofthe claims are embraced by the claims.

What is claimed is:
 1. An image processing device for generating andupdating three-dimensional data representing a biological tissue basedon tomographic data of the biological tissue acquired by a sensorconfigured to acquire the tomographic data while moving through a lumenof the biological tissue, and to cause a display to display thethree-dimensional data as a three-dimensional image, the imageprocessing device comprising: a control unit configured to color, in thethree-dimensional image, at least a voxel representing an inner surfaceof the biological tissue or a voxel that is adjacent to the voxelrepresenting the inner surface and that represents the lumen among afirst voxel group corresponding to a cross section indicated bytomographic data newly acquired by the sensor, in a manner of beingdistinguished from a second voxel group corresponding to another crosssection of the biological tissue.
 2. The image processing deviceaccording to claim 1, wherein the control unit is configured to colorall voxels representing the biological tissue among the first voxelgroup in a manner of being distinguished from the second voxel group. 3.The image processing device according to claim 1, wherein the controlunit is configured to color not only the first voxel group but also atleast a voxel representing the inner surface or a voxel that is adjacentto the voxel representing the inner surface and that represents thelumen among a voxel group corresponding to a cross section adjacent tothe cross section corresponding to the first voxel group, in a manner ofbeing distinguished from a voxel group corresponding to another crosssection of the biological tissue.
 4. The image processing deviceaccording to claim 1, wherein the control unit is configured to set acolor of at least the voxel representing the inner surface or a color ofthe voxel that is adjacent to the voxel representing the inner surfaceand that represents the lumen among the first voxel group to a colordifferent from any color of the second voxel group, thereby coloring atleast the voxel representing the inner surface or the voxel that isadjacent to the voxel representing the inner surface and that representsthe lumen among the first voxel group in a manner of being distinguishedfrom the second voxel group.
 5. The image processing device according toclaim 1, wherein the control unit is configured to cause the display todisplay, together with the three-dimensional image, a two-dimensionalimage representing the cross section indicated by the tomographic datanewly acquired by the sensor.
 6. The image processing device accordingto claim 1, wherein the control unit is configured to combine a firstgraphic element representing a moving range of the sensor with a secondgraphic element representing a position of the sensor and cause thedisplay to display the combination together with the three-dimensionalimage.
 7. The image processing device according to claim 6, wherein thecontrol unit is configured to cause the display to display the firstgraphic element in a direction in which a longitudinal direction of thelumen in the three-dimensional image and a long axis direction of thefirst graphic element are parallel to each other.
 8. An image processingsystem comprising: the image processing device according to claim 1; anda probe including the sensor.
 9. The image processing system accordingto claim 8, further comprising: the display.
 10. An image display methodfor generating and updating three-dimensional data representing abiological tissue based on tomographic data of the biological tissueacquired by a sensor that acquires the tomographic data while movingthrough a lumen of the biological tissue, and causing a display todisplay the three-dimensional data as a three-dimensional image, theimage processing method comprising: coloring, in the three-dimensionalimage, at least a voxel representing an inner surface of the biologicaltissue or a voxel that is adjacent to the voxel representing the innersurface and that represents the lumen among a first voxel groupcorresponding to a cross section indicated by tomographic data newlyacquired by the sensor, in a manner of being distinguished from a secondvoxel group corresponding to another cross section of the biologicaltissue.
 11. The image display method according to claim 10, furthercomprising: coloring all voxels representing the biological tissue amongthe first voxel group in a manner of being distinguished from the secondvoxel group.
 12. The image display method according to claim 10, furthercomprising: coloring not only the first voxel group but also at least avoxel representing the inner surface or a voxel that is adjacent to thevoxel representing the inner surface and that represents the lumen amonga voxel group corresponding to a cross section adjacent to the crosssection corresponding to the first voxel group, in a manner of beingdistinguished from a voxel group corresponding to another cross sectionof the biological tissue.
 13. The image display method according toclaim 10, further comprising: setting a color of at least the voxelrepresenting the inner surface or a color of the voxel that is adjacentto the voxel representing the inner surface and that represents thelumen among the first voxel group to a color different from any color ofthe second voxel group, thereby coloring at least the voxel representingthe inner surface or the voxel that is adjacent to the voxelrepresenting the inner surface and that represents the lumen among thefirst voxel group in a manner of being distinguished from the secondvoxel group.
 14. The image display method according to claim 10, furthercomprising: causing the display to display, together with thethree-dimensional image, a two-dimensional image representing the crosssection indicated by the tomographic data newly acquired by the sensor.15. The image display method according to claim 10, further comprising:combining a first graphic element representing a moving range of thesensor with a second graphic element representing a position of thesensor and cause the display to display the combination together withthe three-dimensional image.
 16. The image display method according toclaim 15, further comprising: causing the display to display the firstgraphic element in a direction in which a longitudinal direction of thelumen in the three-dimensional image and a long axis direction of thefirst graphic element are parallel to each other.
 17. A non-transitorycomputer readable medium storing an image processing program which, whenexecuted by a computer, generates and updates three-dimensional datarepresenting a biological tissue based on tomographic data of thebiological tissue acquired by a sensor that acquires the tomographicdata while moving through a lumen of the biological tissue, and causes adisplay to display the three-dimensional data as a three-dimensionalimage, and performs processing comprising: coloring, in thethree-dimensional image, at least a voxel representing an inner surfaceof the biological tissue or a voxel that is adjacent to the voxelrepresenting the inner surface and that represents the lumen among afirst voxel group corresponding to a cross section indicated bytomographic data newly acquired by the sensor, in a manner of beingdistinguished from a second voxel group corresponding to another crosssection of the biological tissue.
 18. The non-transitory computerreadable medium according to claim 17, further comprising: coloring allvoxels representing the biological tissue among the first voxel group ina manner of being distinguished from the second voxel group.
 19. Thenon-transitory computer readable medium according to claim 17, furthercomprising: coloring not only the first voxel group but also at least avoxel representing the inner surface or a voxel that is adjacent to thevoxel representing the inner surface and that represents the lumen amonga voxel group corresponding to a cross section adjacent to the crosssection corresponding to the first voxel group, in a manner of beingdistinguished from a voxel group corresponding to another cross sectionof the biological tissue.
 20. The non-transitory computer readablemedium according to claim 17, further comprising: setting a color of atleast the voxel representing the inner surface or a color of the voxelthat is adjacent to the voxel representing the inner surface and thatrepresents the lumen among the first voxel group to a color differentfrom any color of the second voxel group, thereby coloring at least thevoxel representing the inner surface or the voxel that is adjacent tothe voxel representing the inner surface and that represents the lumenamong the first voxel group in a manner of being distinguished from thesecond voxel group.